1 Research questions

  • How valid is a measurement model for the perception of 4-factor meritocracy, which is grouped into: a first-order factor related to meritocracy and a second-order factor referred to inequalities of opportunity?

2 General description and hypothesis

To measure the ideals of meritocracy, it is essential to differentiate preferences (i.e. what is considered desirable) and perceptions (i.e. how reality is believed to be) (Duru-Bellat & Tenret, 2012). This work focuses on the perceptions of meritocracy.

Although a group of authors has measured the perception of meritocracy as if the perception of inequality of opportunities were the side opposite of the continuum (Kunovich & Slomczynski, 2007; Newman et al., 2015), others have shown that both concepts are rather different dimensions (Reynolds & Xian, 2014). Following this evidence, we chose to model both constructs as distinct latent variables. Different types of inequalities such as origin, contact or discrimination, cannot be considered in a single factor, since they refer to different realities and perceptions, therefore, it is consistent that they are independent latent variables. We also consider the existence of a latent second order variable, referring to the belief of inequality of opportunities in general.

Consistently, we consider that questions regarding meritocratic preferences will be grouped into two factors, meritocratic preferences and non-meritocratic preferences.

Additionally, it is important to mention that we will exploratory analysis the relationship between the “get a head” questions and questions that refer to whether “In Chile, people are rewarded or manage to succeed according to different qualities (i.e., hardworking, talented, with contacts)”.

  • H1. The perception of meritocracy is a latent variable in the drift of the importance attributed to talent and the effort to get ahead in life.
  • H2. Non-meritocratic perception is a latent variable that derives from two elements related to the agreement with the statement that people with contacts and rich parents, manage to get ahead
  • H3. Meritocratic preferences behave as a latent variable.
  • H4. Non-meritocratic preferences behave as a latent variable.

3 Type of study

The present study is of an observational type focused on the measurement of the meritocracy perception variable. This study is part of an experimental study, applied online, which will be longitudinal panel type, that is, the same subjects will be surveyed 3 different occasions. The subjects will be randomly selected to be assigned to a placebo or to either of the two experimental procedures (change of poverty in Chile or international comparison of inequality) To access the cases we turn to the Netquest survey company, which has coverage in several Latin American countries.

4 Sample

4.1 Detailed data and explanations of its use.

The Sample that will be necessary for this study has not yet been produced, nor has it begun its field work. However, in order to test the online questionnaire and the functionality of the quota offered by the Qualtrics platform, two previous studies have been carried out. The subjects participated in these two opportunities are excluded to the central sample, which corresponds to the 3-wave panel study. These data from the two pilot studies were used for methodological decisions, descriptive statistics, correlations and confirmation tests. Due to the above, although the definitive sample has not yet been produced and therefore has not been explored, it is responsible to say, in addition to the review of previous research and international databases, it has some notions of the relationship among the variables product of the analysis of the previous samples

4.2 Magnitude of the sample and explanation of the magnitude.

A representative sample of the large cities of Chile (Gran Santiago, Valparaíso-Viña del Mar, Concepción and Antofagasta) is expected, which will have an initial number for the first wave of 2100 cases, which are expected to be at least 1800 in the second wave. From the second to the third wave it has been considered as “free fall” that is, we are not sure of the retention rate of the interviewees. The sample size was calculated from power analysis that used information from the pilot study that was applied in June 2019. This pilot study consisted of an online experiment applied to 949 subjects, randomly separating participants to two stimuli and a placebo, leaving a third of the sample in each condition. From these data, the low power of a two-tailed test was calculated, considering an average treatment effect of 0.26, with a standard deviation of the result of the treatment group of 1.23 and a significance level of 0.05. Based on multiple comparison correlations (Bonferroni), we adjusted alpha for six tests (three results * two comparisons between conditions and placebo). The results indicated that the number of subjects exposed to each stimulus should be 580, so we will work with three samples of 600 cases, which gives us a total of 1800 cases for the second wave in which the experiment will be applied. To guarantee the above and in consideration of a high non-response rate, we will start with a sample of 2100 subjects.

4.2.1 Data collection procedure:

To access this target population, a company has been hired dedicated to the online surveys that will be in charge of getting the respondents. These will be invited to participate in the survey under the incentive provided by the external company. Respondents have 4 business days to answer the survey, with a maximum of 20 minutes per survey. In order to achieve greater representativeness of the sample, the quota method will be used, that is, the program will only allow respondents to respond while more people with their demographic characteristics are required. The quotas used were age, sex and education. It should be noted that the time between wave and wave is between 7 and 9 days.

4.2.2 Stop Rule:

The rule of detention is to reach the number of respondents indicated, adjusted to the corresponding quotas. However, there may be modifications depending on the contingencies of the field work, depending on the response rates.

5 Variables to use:

All the variables used in the scale use the following question and answer options

  • To what extent do you agree or in disagree with each of the following affirmations?
    • Strongly disagree
    • Disagree
    • Neither agree nor disagree
    • Agree
    • Totally agree
Table 1. Meritocracy perception and preference
Dimension Factor Statement (english) Statement (spanish)
Perception Meritocratic In Chile, those who work harder achieve greater rewards. En Chile, quienes más se esfuerzan logran obtener mayores recompensas.
In Chile, those who have more talent achieve greater rewards. En Chile, quienes tienen más talento logran obtener mayores recompensas.
Non meritocratic In Chile, people with rich parents succeed. En Chile, las personas con padres ricos logran salir adelante.
In Chile, those who have good contacts succeed. En Chile, los que tienen buenos contactos logran salir adelante.
Preference Meritocratic Those who try harder should get greater rewards than those who work less. Quienes más se esfuerzan deberían obtener mayores recompensas que quienes se esfuerzan menos.
Those with more talent should get greater rewards than those with less talent. Quienes poseen más talento deberían obtener mayores recompensas que quienes tienen menos talento.
Non meritocratic It’s fine that those who have rich parents get ahead. Está bien que quienes tienen padres ricos salgan adelante.
It’s fine that those who have good contacts get ahead. Está bien que quienes tienen buenos contactos salgan adelante.

5.0.1 Variable manipulation

The variables will be worked on the same scale in which they were asked, without resorting to manipulations of the variables to assess the validity of the measurement model.

5.0.2 Indexes

The investigation of the validity of this scale does not resort to indices intended to represent the perception of meritocracy in an observed way, but rather confirmatory factor analysis will be used to estimate the value of the latent variables underlying the indicators.

5.0.3 Blinding of information

No special blinding is used for this study.

5.0.4 Randomization

To avoid the existence of biases in the order of the questions, they have been randomized within a thematic block of questions, so, randomly, people answer questions in different orders regarding their perception of meritocracy.

6 Analysis plan

To evaluate these hypotheses, confirmatory factor analysis (CFA) will be used, since we have strong theory regarding the ordering of the factors. In addition, exploratory factor analysis will be used to evaluate the exploratory hypothesis indicated. The lavaan (Rosseel, 2012) R package will be used.

6.0.1 Inference criteria

The values that will be used as evaluation criteria for the goodness of the adjustment of the model were taken from the proposal of Brown (2008) and are the following:

  • Chi-square:> 0.05
  • Chi-square ratio:> 3
  • Comparative adjustment goodness index (CFI):> 0.95
  • Tucker-Lewis Index (TLI):> 0.95
  • Root of the average squared residual approximation <0.08.

6.0.2 Secondary analysis

To evaluate the metric stability of the measurement model (Davidov et al., 2014) proposed above we will conduct a longitudinal invariance test using the first and third wave of our study. Following Liu et al. (2017), we will test a series of four hierarchical models: Configural, Weak, Strong and Strict invariance models for ordered-categorical indicators based on the assumtion that a five category likert scale cannot be treated as a continous variable because can lead to biased parameter estimates.

6.0.3 Data Exclusion

All cases will be used as long as they do not show missing values in any of the items on the scale. No imputation criteria will be used.

6.0.4 Ethics.

The experiment and survey instruments are approved by the IRB of the University of Chile.

Bibliography

Brown, T. A. (2008). Confirmatory Factor Analysis for Applied Research. https://doi.org/10.1198/tas.2008.s98

Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement Equivalence in Cross-National Research. Annual Review of Sociology, 40(1), 55–75. https://doi.org/10.1146/annurev-soc-071913-043137

Duru-Bellat, A. M., & Tenret, E. (2012). Who’s for Meritocracy? Individual and Contextual Variations in the Faith. Comparative Education Review, 56(2), 223–247.

Kunovich, S., & Slomczynski, K. M. (2007). Systems of distribution and a sense of equity: A multilevel analysis of meritocratic attitudes in post-industrial societies. European Sociological Review, 23(5), 649–663. https://doi.org/10.1093/esr/jcm026

Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506. https://doi.org/10.1037/met0000075

Newman, B. J., Johnston, C. D., & Lown, P. L. (2015). False Consciousness or Class Awareness? Local Income Inequality, Personal Economic Position, and Belief in American Meritocracy. American Journal of Political Science, 59(2), 326–340. https://doi.org/10.1111/ajps.12153

Reynolds, J., & Xian, H. (2014). Perceptions of meritocracy in the land of opportunity. Research in Social Stratification and Mobility, 36, 121–137. https://doi.org/10.1016/j.rssm.2014.03.001

Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48(2), 1–36.